Apply expected consumption for all limits in one write - #60
Conversation
An evaluation that touches two usage limits recorded one of them. _applyExpectedConsumption reads the whole contract, increments a single usage level in its own copy, and writes the whole contract back. evaluateFeature called it once per limit through Promise.all, so every call read the same starting state and only the last write survived - a lost update, silent, on the path that decides whether somebody may use a feature. _applyExpectedConsumptions applies every limit to one contract copy and writes once: correct, and one round trip instead of one per limit. Limits are all validated before anything is written, so naming a limit that does not exist cannot leave the others half-applied. The single-limit method stays, delegating. Seven tests against stubbed repository and cache, including one that pins why the batch method exists: calling the single-limit method concurrently is still lossy, because read-modify-write on a whole document cannot be made safe by calling it more carefully. Making that safe means an atomic $inc in the repository, which is a larger change than this one. Their feature-evaluation suite (26 tests) passes unchanged.
|
The red Integration Tests Run check on this PR is not caused by the change. The job reads its Mongo port and database name from the The workflow has never passed on a fork PR — every green run in its history came from a branch inside the repository. #62 fixes it, and its own check is green, which is the fix running on a fork PR. Once that lands this check should go green here too. |
|
Full suite, verified locally. Since the workflow cannot run on a fork PR until #62 lands, I reproduced the CI environment (Mongo 7.0.16 on 27017, Redis 7 on 6379, an That is the repository's 12 files plus the one this PR adds. The same run on One note on method, since it changed a conclusion: an earlier run of mine reported failures in |
|
Thanks for your contribution. Actually, if you don't mind, would be great to solve the root of the problem by implementing the $inc version |
Batching a whole evaluation into one read and one write stopped an evaluation from losing its own limits, but not two requests from losing each other's: both read the same consumed value, both write the same total, and one consumption disappears. The increment is now handed to Mongo as $inc and evaluated against the stored document, so concurrent calls compose. The filter requires every usage level to exist, which keeps validation and the write in one operation instead of leaving a window between them.
|
Done — this now does the
The test that pinned the lossy behaviour is gone, replaced by three that assert it no longer happens (two concurrent calls on the same limit, twenty concurrent calls, and concurrent calls on different limits). Putting the read-modify-write back fails four of the eleven, so the change is load-bearing. Thanks for pushing for the root fix — it is the better change. |
The problem
An evaluation that touches two usage limits records one of them.
_applyExpectedConsumptionreads the whole contract, increments a single usagelevel in its own copy, and writes the whole contract back:
and
evaluateFeaturecalled it once per limit, concurrently:Every call reads the same starting state, and each writes back its own copy of
the whole document. The last write wins and the other increments are gone.
Why it matters
This is on the path that decides whether somebody may use a feature, and it
fails silently: the evaluation returns
true, the response looks right, and theusage level for every limit but one is simply not moved. Nothing errors, so the
first sign is a quota that never seems to run out.
For a feature with a single usage limit — which is most of them — it never
happens, which is why it can sit unnoticed.
The change
_applyExpectedConsumptions(userId, Record<limitId, amount>)applies everylimit to one contract copy and writes once. Correct, and one round trip instead
of one per limit.
Every limit is validated before anything is written, so a request naming one
limit that does not exist cannot leave the others half-applied — the previous
code could apply some and then throw.
_applyExpectedConsumptionstays, delegating, so nothing that calls it breaks.The root fix, as requested
The first version of this PR batched the limits into one read and one write,
which stopped an evaluation from losing its own limits but left two requests
losing each other's. I flagged that here and @Alex-GF asked for the
$incversion, which is what this now is.
ContractRepository.incrementUsageLevelshands the arithmetic to the database:Two things worth pointing out:
$existsin the filter, not a prior read. Validation and the write are oneoperation, so a limit cannot be checked and then vanish before the update — and
a limit that is not on the contract matches no document instead of being
created by
$inc, which is what would otherwise happen. Which of contract orlimit was missing is worked out afterwards, only to phrase the error, and only
once we know something was.
_revertExpectedConsumptionneeds the value from before this call. Taking it from the result minus the
amount applied means this caller takes back its own contribution even if other
calls landed in between; reading it beforehand would reintroduce exactly the
race being removed.
The service no longer reads the contract at all on this path, so the cached copy
can no longer be the basis of a write — it is only refreshed from the
authoritative result.
Verification
11 tests in
src/test/contract.expected-consumption.test.ts, against astubbed repository and cache so the reads and writes themselves can be asserted.
The stub models the database honestly:
incrementUsageLevelsadds to whatever isstored when it runs, which is the guarantee
$incgives and the one anapplication-side read-modify-write cannot.
Putting the batched read-modify-write back in place of the
$inccall failsfour of them — the three concurrency tests and the round-trip count — so the
change is load-bearing rather than incidental:
Against a real MongoDB 7.0.16, exercising the
$incthrough the API rather thana stub:
Full suite: 13 files, 0 failures.
npx tsc --noEmitis clean.